Repository navigation
csnode: a confidant defers the backfill flush - #147
Merged
Merged
Conversation
On the mainnet test node the backfill flush (transaction index, block hashes, storage) took 358 ms at the median and up to 2.4 s, on the processor thread, every ~25 s. While the node is a confidant it now waits for the first turn outside the trusted set, 10 minutes at most; blocks not flushed yet are fetched again after a crash. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The mainnet test node (full rocksdb from the snapshot, master 8d4bc38) logs each backfill flush with its duration (#140): 69 flushes, median 358 ms, p90 808 ms, max 2398 ms, 6 over 1 s, every ~10,000 blocks (~25 s). The flush runs on the processor thread, outside the turn budget, so on a confidant it can hold consensus for more than a round.
Change
flushIfDue: while the node is a confidant (Context::trusted, #146) a due flush waits for the first turn outside the trusted set, at mostkTrustedFlushDeferMs(10 min) after the last one. A forced flush (end of the backfill, stop) is unchanged. Blocks not flushed yet are fetched again after a crash: the next start resumes from the flushed floor (seen on mainnet today).Limit: a flush started outside the trusted set can still run into a round where the node is trusted; moving the flush to its own thread is a larger change, left for later.
Test
AConfidantDefersTheFlush.Test plan
🤖 Generated with Claude Code